Associative memory of phase-coded spatiotemporal patterns in leaky Integrate and 

Fire networks 
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We study the collective dynamics of a Leaky Integrate and Fire network in which precise relative 
phase relationship of spikes among neurons are stored, as attractors of the dynamics, and selectively 
replayed at different time scales. Using an STDP-based learning process, we store in the connec- 
tivity several phase-coded spike patterns, and we find that, depending on the excitability of the 
network, different working regimes are possible, with transient or persistent replay activity induced 
by a brief signal. We introduce an order parameter to evaluate the similarity between stored and 
recalled phase-coded pattern, and measure the storage capacity. Modulation of spiking thresholds 
during replay changes the frequency of the collective oscillation or the number of spikes per cycle, 
keeping preserved the phases relationship. This allows a coding scheme in which phase, rate and 
frequency are dissociable. Robustness with respect to noise and heterogeneity of neurons parameters 
is studied, showing that, since dynamics is a retrieval process, neurons preserve stable precise phase 
relationship among units, keeping a unique frequency of oscillation, even in noisy conditions and 
with heterogeneity of internal parameters of the units. 
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I. INTRO 

It has been hypothesized that, in many areas of the 
brain, having different brain functionality, repeatable 
precise spatiotemporal patterns of spikes play a crucial 
role in coding and storage of information. Temporally 
structured replay of spatiotemporal patterns have been 
observed to occur during sleep, both in the cortex and 
hippocampus [11-3, and it has been hypothesized that 
this replay may subserve memory consolidation. The se- 
quential reactivation of hippocampal place cells, corre- 
sponding to previously experienced behavioral trajecto- 
ries, has been observed also in the awake state (awake 
replay) j namely during periods of relative immobil- 
ity. Awake replay may reflect trajectories through either 
the current environment or previously, spatially remote, 
visited environments. A possible interpretation is that 
spatiotemporal patterns, stored in the plastic synaptic 
connections of hippocampus, are retrieved when a cue 
activates the emergence of a stored pattern, allowing 
these patterns to be replayed and then consolidated in 
distributed circuits beyond the hippocampus [§]. Cross- 
correlogram analysis revealed that in prefrontal cortex 
the time scale of reactivation of firing patterns during 
post-behavioral sleep was compressed five- to eightfold 
relative to waking state 0, [H , a similar compression ef- 
fect may also be seen in primary visual cortexQ. Inter- 
nally generated spatiotemporal patterns have also been 
observed in the rat hippocampus during the delay pe- 
riod of a memory task, showing that the emergence of 
consistent pattern of activity may be a way to maintain 
important information during a delay in a task [lo| . 

Among repeating patterns of spikes a central role is 
played by phase-coded patterns [lll - fl3H T5| . i.e. patterns 
with precise relative phases of the spikes of neurons par- 
ticipating to a collective oscillation, or precise phases of 



spikes relatively to the ongoing oscillation. 

First experimental evidence of the importance of spike 
phases in neural coding was observed in experiments 
on theta phase precession in rat's place cells [13, [l9j . 
showing that spike phase is correlated with rat's posi- 
tion. Recently, the functional role of oscillations in the 
hippocampal-entorinal cortex circuit for path-integration 
has been deeply investigated [1, [l6l - [l9l , showing that 
place cells and grid cells form a map in which precise 
phase relationship among units plays a central role. In 
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particular it has been shown|2lLl25 
tuning and phase-precession properties of place cells can 
arise when one has interference among oscillatory cells 
with precise phase relationship and velocity-modulated 
frequency. 

Further evidence of phase coding comes from the ex- 
periments on spike-phase coding of natural stimuli in au- 
ditory and visual primary cortex [l^, [l^ , and from ex- 
periments on short-term memory of multiple objects in 
prefrontal cortices of monkeys 

These experimental works support the hypothesis that 
collective oscillations may underlie a phase dependent 
neural coding and an associative memory behavior which 
is able to recognize the phase coded patterns. 

The importance of precise timing relationships among 
neurons, which may carry information to be stored, is 
supported also by the evidence that precise timing of few 
milliseconds is able to change the sign of synaptic plas- 
ticity. The dependence of synaptic modification on the 
precise timing and order of pre- and post-synaptic spiking 
has been demonstrated in a variety of neural circuits of 
different species. Many experiments show that a synapse 
can be potentiated or depressed depending on the precise 
relative timing of the pre- and post-synaptic spikes. This 
timing dependence of magnitude and s ign of p lasticity, 
observed in several types of cortical [sol . liol . |47| | and hip- 
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pocampal [41| - |43l . |47| neurons, is usually termed Spike 
Timing Dependent Plasticity (STDP). 

The role of STDP has been investigated both in super- 
vised learning framework [6^ , in unsupervised framework 
in which repeating patterns are detected by downstream 
neurons [Til, co rtical development [13] , generation of se- 
quences |65l. |66| and polychronous activity (67| . and in an 
associative memory framework with binary units [68l.[69|. 
However, this is the first time that this learning rule has 
been used to make a IF network to work as associative 
memory for phase-coded patterns of spike, each of which 
becomes a dynamic attractor of the network. Notably, 
in a phase coded pattern not only the order of activation 
matters, but the precise spike timing intervals between 
units. 

We therefore present a possibility to build a circuit 
with stable phase relationships between the spikes of a 
population of IF neurons, in a robust way with respect 
to noise and changes of frequency. The first important 
result of the paper is the measurement of the storage 
capacity of the model, i.e. the maximum number of dis- 
tinct spatiotemporal patterns that can be stored and se- 
lectively retrieved, since it has never been computed in a 
spiking model for spatiotemporal patterns. 



Several classic papers (see 5l| and references therein) 
have focused on stora ge c apacity of binary model with 
static binary patterns [37[, and much efforts have been 
done to use more biophysical models and patterns [29l - 
|3i,[3l[5SS[i3,[6i|, but, up to our knowledge, without 
any calculation of the storage capacity of spatiotemporal 
patterns in IF spiking models. Notably, by introducing 
an order-parameter which measures the overlap between 
phase coded spike trains, we are able quantitatively 
measure of the overlap between the stored pattern and 
the replay activity, and to compute the storage capacity 
as a function of the model parameters. 

Another important result is the study of the different 
regimes observed by changing the excitability parameters 
of the network. In particular, we find that near the re- 
gion of the parameter space where the network tends to 
become unresponsive and silent there is a regime in which 
the network responds selectively to cue presentation with 
a short transient replay of the phase-coded pattern. Dif- 
ferently, in the region of higher excitability, the patterns 
are replayed persistently and selectively, and eventually 
with more then one spike per cycle. 

The paper is organized as follows: Sectionllllintroduccs 
the Leaky-Intcgratc-and-Fire (IF) neuronal model; Sec- 
tion mil describes the STDP learning rule used to design 
the connections; in Section IIVI we study the emergence 
of collective dynamics and introduce an order parameter 
to measure the overlap between the collective dynamics 
and the stored phase coded patterns; Section IVl reports 
on the storage capacity of the network, i.e. the maximum 
number of patterns that can be stored and selectively re- 
trieved in the network; the parameter space and the dif- 
ferent working regions arc also investigated in Section [V] 



in Section we study the robustness of the retrieval dy- 
namics wrt noise and heterogeneity; Section IVlII reports 
on the implication of this model in the framework of oscil- 
latory interference model of path-integration; summary 
and discussion are outlined in Section IVIIIl 



II. THE MODEL 

We consider a recurrent neural network with N{N — 1) 
possible connections J^- , where N is the number of neural 
units. The connections are designed during the learn- 
ing mode, when the connections change their efficacy ac- 
cording to a learning rule inspired to the STDP. After 
the learning stage, the connections values are frozen, and 
the collective dynamics is studied. This distinction in two 
stages, plastic connection in the learning mode and frozen 
connections in the dynamics mode, is a useful framework 
to simplify the analysis. It also finds some neurophys- 
iological motivations in the effects of neuromodulators, 
such as dopamine and acetylcholine (HsI . [soj , which regu- 
late excitability and plasticity. 

The single neuron model is a Leaky Intcgrate-and-Fire 
(IF) [ssj . This simple choice, with few parameters for 
each neuron, is suitable to study the emergence of collec- 
tive dynamics and the diverse regimes of the dynamics, 
instead of focusing on the complexity of the neuronal 
internal structure. We use the Spike Response Model 
(SRM) formulation [H [H] of the IF model, which al- 
lows us to use an event-driven programming and makes 
the numerical simulations faster with respect to a differ- 
ential equation formulation. 

In this picture, the postsynaptic membrane potential 
is given by: 



(1) 



where are the synaptic connections, e{t) describes the 
response kernel to incoming spikes on neuron i, and the 
sum over tj runs over all presynaptic firing times follow- 
ing the last spike of neuron i. Namely, each presynaptic 
spike j, with arrival time tj, is supposed to add to the 
membrane potential a postsynaptic potential of the form 
Jijt{t — tj), where 



e{t-i,) = K 



exp 



t-t. 



t-t. 



' exp y i j e(t-fj) 

(2) 

where t„i is the membrane time constant (here 10 ms), 
Tg is the synapse time constant (here 5 ms), Q is the 
Heaviside step function, and K is a multiplicative con- 
stant chosen so that the maximum value of the kernel is 
1 . The sign of the synaptic connection Jjj sets the sign of 
the postsynaptic potential's change, so there's inhibition 
for negative Jij and excitation for positive Jij . 

When the membrane potential hi{t) exceeds the spik- 
ing threshold 9\f^, a spike is scheduled, and the membrane 
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potential is reset to the resting value zero. We use the 
same threshold 9th for all the units, except in sec |VI] 
where different values Oly^ are used and the robustness 
w.r.t. the heterogeneity is studied. Clearly the spiking 
threshold 0th of the neurons is related to the excitabil- 
ity of the network, an increase of the value of 9th is also 
equivalent to a decrease of K, the size of the unitary post- 
synaptic potential, or, cquivalcntly to a global decrease 
in the scaling factor of synaptic connections Jy . 
Numerical simulations of this dynamics are performed for 
a network with P stored patterns, where connections Jij 
are determined via a learning rule described in the next 
paragraph. We found that a few number of spikes, given 
a in proper time order, are able to selectively induce the 
emergence of a persistent collective spatiotemporal pat- 
tern, which replays one of the stored pattern (sec sec II VI) . 



DESIGNING THE CONNECTIONS OF THE 
NETWORK 



III. 



In a learning model previously introduced in [491 - 15 Ij . 
the average change in the connection J^ , occurring in 
the time interval [— ifearnjO] due to periodic spike trains 
of period T, with tieam >> T, was formulated as follows: 



SJij = 



T 



dt 



dt' x^{t)A{t~t')xJ{t') (3) 



where T/tieam is a normalization factor, Xj(t) is the ac- 
tivity of the pre-synaptic neuron at time t, and Xi(t) the 
activity of the post-synaptic one. It means that the prob- 
ability for unit i to have a spike in the interval (t, t + At) 
is proportional to Xi{t)At in the limit At — 0. The learn- 
ing window A(r = t ~ t') is the measure of the strength 
of synaptic change when a time delay r occurs between 
pre and post-synaptic activity. To model the experimen- 
tal results of STDP in hippocampal neurons, the learn- 
ing window A(t) should be an asymmetric function of 
r, mainly positive (LTP) for r > and mainly negative 
(LTD) for T < 0. 

Equation (jSj holds for activity pattern x{t) which rep- 
resents instantaneous firingrate, and is suitable to use 
in analog rate models |49l453j and spin network models 
[H, [gI]. Differently, here, being interested in spiking 
neurons, the patterns to be stored are defined as precise 
periodic sequence of spikes, i.e. spike-phase coded pat- 
terns. Namely, activity of the neuron j is a spike train 
at times tj, 



time duration of the learning process ti^arn is longer then 
a single period T'', is simply given by 



Ait^-tf + nTf"). 



The window A(t), shown in Fig. 1, is given by 

for T > 



A{t) 



-/Tn 



for T < 0, 



(5) 



(6) 



with the same parameters used in [54| to fit the exper- 
imental data of [4l|, namely ap = -f[l/Tp + t^/TdJ^^, 
ao =J [v/Tp + 1/Td]-^, with Tp = 10.2 ms, Td = 28.6 
ms, 77 = 4, 7 = 0.42. This function satisfies the balance 
condition A{T)dT = 0. Notably, when A{t) is used 
in eq. [5] to learn phase-coded patterns with uniformly 
distributed phases, then the property J A{T)dt = as- 
sures that in the connection matrix the summed ex- 
citation (1/-^) X^i J >o "^^i summed inhibition 
i,Ji <o "^ij equal in the thermodynamic limit, 
and therefore it assures a balance between excitation and 
inhibition. 

Writing Eq. (I5][5]), implicitly we have assumed that, 
with periodic phase-coded spike trains used to induce 
plasticity, the effects of all separate spike pairs sum lin- 
early, each weighted by the same STDP window reported 
in Fig. 1. Timing-dependent learning curves as the one 
reported in Fig. 1 are indeed typically measured by giv- 
ing an order of 100 pairs of spikes repeatly, with fixed 
phase relationship, and fixed frequency in a proper range. 

However, in different situations, for instance if the fre- 
quency is too low or to high H^j , or in case of few spike 
pairs [27[ , the timing dependence of plasticity is not well 
described by the bidirectional window used here, and a 
more detailed model is needed to account for integration 
of spike pairs when arbitrary trains are used (see [U l45j 
and references therein). 




50 100 

T (ms) 



^(t)=^<5(t-(t;+nr'')). 



(4) 



where tj + nT'^ is the set of spikes times of unit j in the 
pattern fi with period T^, and frequency i/^^ = \/T^. 
Therefore, following Eq.®, the change in the connec- 
tions Jij due to the learning of the pattern ^ when the 



FIG. 1: a) Plot of the learning window A(t) used in the 
learning rule (see Eqs. (3), (5), (6)) to model STDP effects. 
The parameters of the function A{t) (Eq. (6)) are determined 
by fitting the experimental data reported in [4H . 

The spikes patterns used in this work are periodic spa- 
tiotemporal sequences, made up of one spike per cycle 
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and each of which has a phase randomly chosen from 
a uniform distribution in [0,27r). In each pattern, infor- 
mation is coded in the precise time delay between spikes 
of unit i and unit j, which corresponds to a precise phase 
relationship among units i and j. A spatiotemporal pat- 
tern represented in this way is often called phase coded 
pattern. Pattern's information is coded in the spiking 
phases which, in turn, shape the synaptic connectivity 
responsible of the emerging dynamics and the memory 
formation. 

The set of timing of spikes of unit j can be defined as 
4- nT'" = ((?!)p/(27rj/^) + n/j/^, where v'^ is the oscil- 
lation frequency of the neurons. Thus, each pattern fi 
is represented through the frequency v'^ and the specific 
phases of spike (f>j of the neurons j = 1,..,N. The change 
in the connection Jij provided by the learning of pattern 
fi is given by 



(7) 

When multiple phase coded patterns are stored, the 
learned connections are simply the sum of the contri- 
butions from individual patterns, namely 



(8) 



Note that ring-like topology with strong unidirectional 
connections is formed only in the case P=l, when a sin- 
gle pattern is stored. When multiple patterns are stored 
in the same connectivity, with phases of one pattern un- 
correlated with the others, bidirectional connections are 
possible, and the more the stored patterns, the less the 
ring-like is the connectivity. Even in the cases when the 
connectivity is not ring-like the network is still able to 
retrieve each of the P stored patterns in a proper range 
of threshold values (see storage capacity in Sec IV)) . 



IV. EMERGING OF COLLECTIVE PATTERNS 
IN THE NEURAL DYNAMICS OF THE 
NETWORK 

We study a recurrent network with TV leaky Integrate 
and Fire units, with connections fixed to the values cal- 
culated in Eqs. (7,8) for different values of P. The results 
show that, within a well specified range of parameters, 
our IF network is able to work as an associative memory 
for spike-phase patterns. 

In order to check if the network is able to retrieve se- 
lectively each of the stored patterns, we give an initial 
signal, made up of M <^ N spikes, taken from the stored 
pattern /x, and we check if this initial short cue is able 
to selectively trigger a collective sustained activity that 
is the replay of the same stored pattern /i, i.e. checking 
if the sustained activity has spikes aligned to the phases 
(?5>f of pattern fi. 



An example of successful selective retrieval process is 
shown in Fig. 2 where, depending on the partial cue 
presented to the network, a different collective activity 
emerges with the phases of the firing neurons which re- 
semble one or another of the stored patterns. 
In this work the cue is a stimulation with M spikes, with 



M 



N/W, at times if = T< 



< i < M, with 



Tstim = 50ms. In the example shown in Fig. 2 the short 
stimulation (which lasts less then 5 ms, shown in pink 
in all the figures) has the effect to selectively trigger the 
sustained replay of pattern /i. 

Note that the retrieval dynamics has the same phase re- 
lationship among units than the stored pattern, but the 
replay may happen on a time scale different from the scale 
used to store the pattern, and the collective spontaneous 
dynamics is a time compressed (or dilated) replay of the 
stored pattern. Indeed, the period of the collective pe- 
riodic pattern which emerges during retrieval stage may 
be different then the period of the periodic pattern used 
in the learning stage. In the example of Fig. 2 the time 
scale of the retrieval dynamics (Fig. 2c, d) is faster then 
the time scale used to learn the patterns (Fig. 2a, b). In 
the following we will study the factors affecting the time 
scale during retrieval, given the time scale of the pattern 
used during learning. 

Clearly, regions of the parameter space in which the net- 
work is unable to retrieve selectively the patterns also 
exist. In these regions the retrieval dynamics may cor- 
respond to a mixture of patterns or a spurious state, i.e. 
a state which is not correlated with any of the stored 
patterns because the number of stored patterns exceeded 
the storage capacity of the network. As discussed below, 
the storage capacity, defined as the maximum number of 
encoded and successfully retrieved patterns, depends on 
the frequency used during the learning stage (which af- 
fects connectivity), and on the spiking threshold of the 
units (which affects excitability and network dynamics). 

Example of failure are shown in Fig. 3. In Fig. 3b the 
network has too low excitability and the response is not 
persistent, while in Fig. 3a the emerging dynamics is not 
correlated with any of the stored patterns. 
To measure quantitatively the success of the retrieval, in 
analogy with the Hopfield model, we introduce an order 
parameter, which estimates the overlap between the net- 
work collective activity during the spontaneous dynamics 
and the stored phase-coded pattern. This quantity is 1 
when the phases of neurons j coincides with the stored 
phases (j)^ , and is close to zero when the phases are un- 
correlated with the stored ones. Therefore, we consider 
the following dot product \m>^{t)\ =< £.{t)\^^ > where 
is the vector having components e"^3 , namely: 



\m^{t)\ 



- E 



j=l..-.,N 
t-T* <£* <t 



(9) 



where t* is the spike timing of neuron j during the 
spontaneous dynamics, and T* is an estimation of the 
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FIG. 3: Example of neural response in two case of failure 
of retrieval. A spurious state emerge in (a), while a short 
transient response emerges in the case shown in (b). A'' = 
3000 and v'^ = 3Hz as in previous picture, while the values 
6th and P are 9th = 10, P = 5 in (a) and dth = 95, P = 5 in 
(b). The dynamics emerging after a short train of M — TV/ 10 
spikes with phases equal to the stored pattern (pattern shown 
in Fig 2a), is not a self-sustained retrieval of the pattern. For 
clarity, the raster plot of only 50 (randomly chosen) units are 
shown, sorted according to increasing value of phase (/>i^ of 
the stored pattern. 



FIG. 2: Examples of selective successful retrieval (c,d) of two 
stored patterns (a,b). The raster plot of 50 units (randomly 
chosen) are shown sorted on the vertical axis according to in- 
creasing values of phase (j)} of the first stored pattern jj. — 1. 
The network has N = 3000 IF neurons, Oth ~ 70 and connec- 
tions given by Eq. (7,8) with P = 5 stored patterns at i^** = 3 
Hz. Two of the stored patterns used during the learning mode 
are shown in a,b. The dynamics emerging after a short train 
of M = N/10 spikes with phases similar to the pattern shown 
in a,b, is shown in c,d respectively. The dynamics of the net- 
work, after a transient, is periodic of period T. The spikes 
which belong to the trigger are shown in pink in (c,d), the 
other different colors represent the value oiti/T modA, where 
ti is the time of the spike of the unit i during the emerging 
spontaneous dynamics. Figure c shows that when the network 
dynamic is stimulated by a partial cue of pattern /i = 1, the 
neurons oscillate with phase alignments resembling pattern 
= 1, but at different frequency. Otherwise, in d, when the 
partial cue is taken from pattern /i = 2, the neurons phase 
relationships, even if periodic, are uncorrelated with pattern 
/X = 1, and recall the phase of pattern fj, — 2. 



period of the collective spontaneous periodic dynamics. 
The overlap in Eq. ^ is equal to 1 when the phase- 
coded pattern is perfectly retrieved (i.e. same sequence 
and phase relationships among spikes, even though on 
a different time scale), while is of order ~ l/vW when 
phases of spikes are uncorrelated to the stored phases. 
The order parameter |m^| allow us to measure the 
network storage capacity in the space of parameters 9th 
and z/^. 

Note that the value of (t) between two periodic spike 
trains measures the similarity in the sequence of spiking 
neurons and in the phase lag between the spikes, being 



invariant by a simple change in time scale. This is a 
suitable choice especially when the replay of a spatio 
temporal pattern has to be detected independently from 
the compression of the time scale. Note that if we have 
a spike train that is not periodic, we cannot define the 
period, however we can define the order parameter ^ 
looking at the time-window T* which maximize the 
order parameter. This can be useful in the case when 
one looks for a short replay hidden in a not-periodic 
spike train, such in many experimental situations. 
The value of {t) after a transient converges to a stable 
value which is close to one when pattern /i is retrieved 
(for example in Fig. 2c at large times m^^^ = 1, and 
m^^^ = 0.01) while m^^t) is of order ~ I/^/N for all fi 
in the case of failure of retrieval. Two further cases of 
failure can occur: in Fig. 3a m'^(t) after the transient 
has values in the range 0.01 — 0.02 for all /x because the 
emerging dynamics is a spurious state not correlated 
with any of the stored phase-patterns, while m^(i) is 
zero in Fig. 3b since the network becomes silent. 
In the following, the storage capacity of the model is ana- 
lyzed considering the maximum number of patterns that 
the network is able to store and selectively recall. In par- 
ticular we investigate the role of two model parameters: 
the frequency of the stored patterns t'p, and the spik- 
ing threshold dth affecting the excitability of the network. 



V. STORAGE CAPACITY 

Numerical simulations of the IF network with 
N — 3000 neurons were performed by systematically 
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changing the value of the spiking threshold 0th, the 
connections Jij, and for different number of patterns 
P and frequency Vf^. Here we propose results for a 
unique value of the spiking threshold 9th for all neurons, 
however the behavior is also robust with respect to a 
variability in the threshold values among neurons, as 
reported in the next Section. 

Network storage capacity is defined as ac — Pmax/N , 
where N is the number of neurons and Pmax is the 
maximum number of patterns that can be stored and 
successfully retrieved with an overlap |m^| larger than a 
certain value, which measures the degree of similarity. 
Given that in our simulations the overlap |m''(t)| at 
large times has mostly two possible values, close to one 
(success) or close to 1/ y/N (failure) , we fixed the desired 
similarity value to 0.5, since the whole storage capacity 
analysis is very robust with respect to this parameter 
(since the transition between low values and high values 
of 1 771^(^)1 as a function of P is sharp). 
Patterns with random phases were extracted and used to 
define the network connections Jij with the rule Eq. ([5]) . 
After the stimulation with a short train of M = 
spikes taken at times ti from the first pattern, the 
dynamics is simulated and the overlap defined in Eq. 
^ with ^ = 1 is evaluated at large times. If the overlap 
|m''^^(t)|, averaged over 50 runs, is greater than 0.5 at 
time t > t (where t = 600 ms for all the simulations), 
then we consider the retrieval successful for that pattern. 
The maximum value of P, for which the network is able 
to successfully replay each of the stored patterns, defines 
the storage capacity of the network. 
The storage capacity as a function of the spiking thresh- 
old 9th and storing frequency v'^ is reported in Fig. 2^, 
where Pmax is shown in a color-coded legend. The largest 
capacity is achieved when the frequency of the stored 
patterns during learning is v'^ ~ 8 Hz and the spiking 
threshold of the units during retrieval is 9th — 130, 
which provides a capacity amax = Pmax/N = 0.016. 
In Fig. |3b we show the storage capacity Pmax as a 
function of the frequency once fixed the value of the 
threshold 9th to the optimal value, corresponding to 
highest capacity for each frequency. 
The optimal storing frequencies and threshold values 
depend on the time constants of the model, such as the 
Ts,Tm of the IF units and the temporal shape of the 
learning kernel A{t), whereas different shapes of A{t) 
may subserve to different storing frequency ranges. In 
this work r^, Tm and A(t) are set to the values described 
in Sec. and the emergent collective dynamics is 

studied as a function of the other network parameters. 
Indeed, Fig. |4}d shows that for the learning kernel A{t) 
used here, there is peak in the storage capacity around 8 
Hz, in the range 2Hz-20Hz. Figure 0^ also proves that, 
for each stored frequency, a large interval of spiking 
threshold values 9th exists for which the network is 
still able to work properly as associative memory for 
phase-coded patterns. 

The associative memory properties as a function of 
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FIG. 4: (a) Storage capacity in a network oi N = 3000 units, 
as a function of the spiking threshold 9th and oscillation fre- 
quency of stored patterns. The maximum number of pat- 
terns successful retrievable Pmax is shown in color-coded leg- 
end, the value grows from Pmax = (dark blue) to Pmax ~ 50 
(strong red), (b) The storage capacity Pmax as a function of 
the frequency of stored patterns, once fixed the threshold 6th 
to the optimal value for each frequency. 

the spiking threshold are reported in Fig. [5l when 
the oscillation frequency of the patterns stored during 
learning is j/^ = 3Hz. The region marked in green in 
Fig. [5] corresponds to cases in which the retrieval is 
successful and the cue is able to selectively activate 
the self-sustained replay of the stored pattern (with an 
order parameter larger than 0.5). When spiking 
threshold changes in the range 10 < 9th < 90 the 
storage capacity changes between Pmax/N = 1/3000 
and Pmax/N = 29/3000. 

Outside the green region the number of patterns exceeds 
the storage capacity and the retrieval fails. There 
are two possible reasons for this behavior. At low 
threshold, when the number P of patterns exceeds the 
storage capacity Pmax, the network responds with a 
self-sustained activity that is not correlated with any of 
the stored patterns, i.e. a spurious state. In this regime, 
marked with red color in Fig. 5a, the order parameter 
ra^{t) is of order 1/\/N for all the stored patterns (see 
also raster plot in Fig. 3a). On the contrary, in the high 
9th regime, the network tends to become silent and 
unresponsive. Indeed, in the region marked with blue 
color, the network responds to the initial cue stimulation 
with a short transient and then became silent. In this 
case the value of m^{t) is zero because there is no 
self-sustained activity at time t > t, meaning that the 
stored attractors become unstable when 9th is too high 
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FIG. 5: a) Storage capacity at i^^ = 3 Hz: the region of suc- 
cessful retrieval as a function of spiking threshold and number 
of patterns is marked in green. The region with persistent ac- 
tivity not correlated with any of the stored pattern is marked 
in red (spurious states), and the region in which the network 
responds with only a short transient and then becomes silent 
is marked in blue (see examples in fig 3). b) The probability 
that the size Stot of the network response, measured as the 
number of the spikes that follow the cue stimulation, is larger 
than n, with n = 0, N/2, N, is shown as a function of spiking 
threshold 0th, in a network with — 3Hz and P = 1. As 
always in this paper the number of units is A'^ = 3000. The 
figure shows that near Sj"' — 90 there's a transition from a 
region of persistent replay to a region of silence. 



(see raster plot in Fig. 3b). 

For values of the threshold greater than 6*^^**, indepen- 
dently from P, the network activity is never persistent, 
as reported in Fig. [S^ where 0^"* = 90 . 
At thresholds close to this critical value the network 
responds with a transient activity that is a short replay 
of the stored pattern, but not a persistent replay. The 
size Stot of the network response, measured as the 
number of spikes that follow the cue stimulation, is 
reported in Fig. [5]3 as a function of 9th, for a network 
with zy^ = 3 Hz, iV = 3000 and P = 1. 
In the following we investigate the replay activity in 
the region with successful retrieval. We focused on the 
dependence of the frequency of collective oscillations 
during replay on the model parameters. Fig. shows 
the collective frequency of replay as a function of the 
frequency ly'^ of the patterns stored in the learning stage 
with N:=3000. The red dots in the figure refer to the 
frequencies of oscillations observed during retrieval at 
the optimal spiking threshold (where the maximum 
storage capacity occurs), while the bar indicates the 
available range of frequencies of replay, accessible 
through a change in the spiking threshold. Important to 
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FIG. 6: (a) Frequency of the collective dynamics during 
replay as a function of the frequency of stored patterns in the 
network with N=3000 units. Dots refer to replay frequency 
observed at optimal spiking threshold. The bars refers to 
the range of frequency available through changes in spiking 
threshold, (b) Frequencies of the collective dynamics during 
replay as a function of the spiking threshold and for different 
stored frequencies (see colors legend). Pattern is replayed on 
a time scale which becomes faster if we decrease threshold 9th 
for the most of the frequencies i^^. The dependence is much 
stronger for i/*^ < 4 Hz. N=3000. (c) The number of spikes 
per cycle as a function of the spiking threshold 6th in networks 
with different frequency v'^ of stored pattern, (d) Frequency 
of the dynamics during replay as a function of ratio between 
spiking threshold and network size N. Red dots are results for 
a network with N= 10000 units, while blue squares are results 
for a network with N=1000 units. Size of the symbols refers 
to the stored frequency, small symbols (on the top of the 
picture) correspond to stored frequency = lOHz, medium 
size symbols correspond to stored frequency u'^ = 3Hz, and 
large symbols (bottom) correspond to stored frequency v'^ = 
IHz. 



note is that, in most of the cases, the frequency of the 
stored pattern and the collective replay frequency do 
not coincide, since the pattern is replayed compressed 
(or dilated) in time, on a time scale dependent on the 
network parameters. We observe that for the chosen 
parameters Tm,Ts of the network, and the given shape 
of A{t), the replay occurs on a compressed time scale 
for all stored patterns of frequency lower then 25 Hz, 
while the two time scale coincide when j/^ ~ 25 Hz. The 
dependence of the frequency of the collective oscillations 
on the spiking threshold is shown in Fig. [SJd. This 
dependence is weak for stored frequencies higher than 
10 Hz. Besides, for low stored frequencies (1-4 Hz) the 
frequency of the replay is very sensitive to the threshold 
value, changing from 6 Hz at high spiking threshold to 
30 Hz at low threshold. 

We also investigate the frequency's dependence on 
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network size N. In fig. |6ld red dots are results for a 
network witli N ~ 10000 units, while blue squares are 
results for a network with = 1000 units. If what 
counts is only the time lag between the single units 
consecutive in the sequence, then one expects that result 
with IHz stored at iV = 10000 would be similar to lOHz 
stored at = 1000, and this is not the case. We see 
that when we use a storage frequency equal to 10 Hz 
(which corresponds to different time lag between cells 
depending on N), then the oscillation frequency during 
replay is around 30Hz in both networks (both N = 1000, 
and N = 10000), while, on the other hand, if we have a 
storage frequency equal to 1 Hz the oscillation frequency 
during replay may span a large range (5Hz -25Hz) in 
both networks. Fig. Eld also shows that frequency of 
replay depends on the ratio between spiking threshold 
and network size N, and that the high sensitivity on 
spiking threshold value holds, when stored frequency is 
low (1-4 Hz), also at different values of the network size. 
This open the possibility to govern the oscillation 
frequency of the collective replay activity via neuromod- 
ulators which change the excitability and therefore the 
spiking threshold of the neurons. Since in our model 
(see Eq. 1,2) a change of the threshold is equivalent to 
a change in the scale factor of all synaptic connections, 
a similar effect might be achieved also by simply driving 
the cells more due to increased synaptic input. Impor- 
tantly, the sensitivity of collective oscillation frequency 
on spiking threshold is not a sensitivity of the single unit 
but of the collective behavior, since, as discussed in Sec. 
IVIl if we change the spiking threshold of few units the 
collective rhythm is still unique for the whole population. 
The replay frequency depends on the average threshold 
among units, but all the units have the same oscillation 
frequency during replay. 

Moreover, for networks with u'^ > 10 Hz, whose replay 
frequency does not considerably change with spiking 
threshold, the replay dynamics is still affected by the 
spiking threshold. Indeed, in this case, the number of 
spikes per cycle increases with lowering of the spiking 
threshold. An example is reported in Fig. [T] The raster 
plots show the same pattern replayed in three networks 
having different values of the spiking threshold 9th- a 
burst of activity takes place within each cycle, with 
phases aligned with the pattern, with a number of spikes 
per cycle dependent on the value of 9th- This behavior 
is summarized in Fig. 6c where the number of spikes 
per cycle is reported as a function of spiking threshold, 
at different values of stored frequencies. Therefore, by 
lowering the spiking threshold the replay activity occurs 
with more than one spike per cycle, or on a faster time 
scale (see Fig. 6b, c). 

The behavior of the output oscillation frequency 
suggests that a parameters region exists where the net- 
work always responds with one spike per cycle. In this 
region an increase of the excitability produces a growth 
of the frequency of oscillation up to a plateau value. 
Differently, for higher excitability the frequency does 
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FIG. 7: Modulation of spiking threshold changes the number 
of spikes per cycle, keeping preserved the phase relationship 
among units. Recall of the pattern fj, = 1 for networks of 
= 3000 units, u'^ = 20 Hz and different values of spiking 
threshold 9th = 80, 65, 40 is shown respectively in a,b,c. De- 
pending on the value of the spiking threshold 6th, the phase- 
coded pattern is replayed with a different number of spikes per 
cycle. Spike of the cue stimulation are shown in pink, while 
the response of the network in black. For clarity, the raster 
plot of only 50 (randomly chosen) units are shown, sorted ac- 
cording to increasing value of phase (f)i^ of the stored pattern. 



not increase while the number of spikes per cycle grows. 
This means that the different frequencies, in addition 
to the information coded in the phase relationship, can 
code other information in relationship with the level 
of spiking threshold: at high frequency the threshold 
changes the number of spikes per cycle, while at low 
frequency the threshold changes the frequency of the 
collective oscillations during the replay. 
This open the possibility to have a coding scheme in 
which, while the phases encode pattern's information, 
a change in frequency or a change in rate in each cycle 
represents the strength and saliency of the retrieval or 
it may encode another variable [601 ■ The recall of the 
same phase-coded pattern with a different number of 
spikes per cycle is particularly interesting at the light of 
recent observations of Huxter et al. [20j in hippocampal 
place cells, showing the occurrence of the same phases 
with different rates. The authors prove that the phase 
of firing and firing rate are dissociable and can represent 
two independent variables, e.g. the animal location 
within the place field and its speed of movement through 
the field. 

Notably, the recall of the same phase coded pattern with 



different frequencies of oscillation is also relevant and 
accords well with the need to have stable precise phase 
relationship among cells with frequency of oscillation 
modulated by parameters such as the speed of the 
animal [1^ [Ij] ■ 

The value of the frequency of collective activity during 
the replay clearly is related not only to the threshold and 
the stored frequency, but also to the shape of the learning 
window A{t) and on the two characteristic times of the 
model Tg, T,„. A systematic study of the dependence of 
the replay time scale on the shape of STDP and the char- 
acteristic times of the neuron model has not yet done in 
a spiking model, however a dependence on the asymme- 
try of A(t) has been analytically found in a simple model 
with analog neurons and a single characteristic time [i^ . 



VI. EFFECTS OF NOISE AND ROBUSTNESS 
OF COLLECTIVE OSCILLATION FREQUENCY 
AND PHASE RELATIONSHIPS 

While in the Hopfield model the patterns are static, 
and information is coded in a binary pattern ~ 
Si , . . . , S'j!^, with 5f G {±1}: here, in this study, the 
patterns are time dependent, and information is coded 
in the phase pattern (f)'^ = (j)^, with 0f € [0, 27r], 

where the value 0f/(27ri/'') represents the time shift of 
the spike of unit i with respect to the collective rhythm, 
i.e the time delay among units. However, as for the Hop- 
field model, the patterns stored in the network are at- 
tractors of the dynamics, when P do not exceeds storage 
capacity, and the dynamics during the retrieval is robust 
with respect to noise. We firstly check robustness w.r.t. 
input noise, i.e when a Poissonian noise rii{t) is added to 
the postsynaptic potential hi{t) given in EqU]. The total 
postsynaptic potential of each neuron i is then given by 



h^{t) = rnit) + - ^^■) (10) 



where r]i(t) is modelled as 

— *^noisc ^ ^ ^(^ ^noisc)- 



(11) 



t„oiso>ti 



The times inoisc are randomly extracted for each neuron i, 
and Jnoise are random strengths, extracted independently 
for each neuron i and time inoise- The intervals between 
times tnoise are extracted from a Poissonian distribution 
P{St) oc e"''*/^''^'^"™'"^ while the strength Jnoise is ex- 
tracted from a Gaussian distribution with mean Jnoise 
and standard deviation cr (Jnoise )■ 

The network dynamics during the retrieval of a pattern 
in presence of noise is shown in Fig. |S]with different levels 
of noise (rnoise = 10ms, J = and cr( Jnoise) = 0, 10, 20, 30 
in a,b,c,d respectively). Results show that when the noise 
is not able to move the dynamics out of the basin of at- 
traction, the errors do not sum up, and the phase rela- 
tionship is preserved over time (see Fig. [B^,b,c). If the 




FIG. 8: (a,b,c,d) Robustness wrt noise. Raster plots show 
that, when the pattern retrieval is triggered, network's spikes 
continue to have phase alignments resembling the pattern 
even in presence of noise. Errors do not sum up until the 
system is in the basin of attraction of the phase-coded pat- 
tern, as in a,b,c. Different levels of noise are used in a,b,c,d 
(""(Jnoise) ~ 0,10,20,30 respectively), and pattern is trig- 
gered with M = A'^/10 as in previous cases. Only in d the 
level of noise is too high and the system goes out of the basin 
of attraction. (e,f) For comparison, the dynamics, when the 
retrieval is not triggered {M = 0), is shown in subplot e,f in 
presence of the same noise used in c,d. Figure e shows that 
the noise used in c usually affects strongly the dynamics of 
the network, however if the collective oscillation is retrieved 
the system is robust wrt noise. Thresholds in all figures are 
dth ~ 80, A'' = 3000, and synaptic connections Jij are build 
learning P — 2 phase-patterns at uj'^ = 3IIz. 



input noise is very high, as in the example of Fig. [8jl, 
the dynamics moves out of the basin of attraction. 
In order to see the effects of input noise level used in Fig 
[8j;d, we report in Fig. |8^f the network dynamics when 
the pattern retrieval is not initiated (M = 0). In partic- 
ular. Fig. ^ shows that the noise level used in Fig. [5}; is 
strong enough to generate spontaneous random activity 
in absence of the initial triggering, but is not sufficient 
to destroy the attractive dynamics during a successful 
retrieval. As in the Hopfield model, errors do not sum 
up and the dynamics spontaneously goes back to the re- 
trieved phase-coded pattern for all the perturbations that 
leave the system inside the basin of attraction. 

Lastly, the robustness of retrieval w.r.t. heterogeneity 
of the spiking thresholds is investigated. This analysis 
can be carried out by using a different value of spiking 
threshold for each neuron i: 



(1 + zCi, 



yth 



(12) 



where Qi is a random number extracted from a uniform 
distribution in [—1, 1], and z is the degree of heterogene- 
ity. Even with high degree of heterogeneity, the emer- 
gence of the retrieval collective dynamics forces all neu- 
rons to have exactly the same frequency of oscillation 
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and to keep a precise phase relationship, in a very robust 
manner. Figure [5^, b shows the dynamics with threshold 
heterogeneity z = 0.2,0.5 respectively, while the remain- 
ing parameters are set to the values of Fig. [5^. 
The above analysis shows that a unique collective fre- 
quency emerges, which is the frequency corresponding 
to the mean value of the 6'J,j. This is also evident by 
comparing Fig. |n^,b with Fig. [S^. At 2; = 0.5 it can 
be seen an additional small phase shift proportional to 
the value of Q of each neuron, but collective activity is 
preserved. Clearly if z is too high and threshold values 
are distributed out of the region of successful retrieval 
the network is unable to retrieve the pattern and failure 
happens, as already discussed in Sec. |Vl 
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FIG. 9: Robustness wrt heterogeneity of spiking threshold 
values. Raster plots show that, when the pattern retrieval is 
triggered, units participate to the network collective oscilla- 
tion, showing all the same frequency and phase alignments 
resembling the pattern, even in presence of threshold values 
heterogeneity among units. Spiking thresholds of neuron i 
are distributed according to = Qth(X + ^C) with average 
Qth ~ 80 and z = 0.2, 0.5 in (a,b) respectively. All other pa- 
rameters are as in Fig. 8a ( N=3000, M=N/10, and synaptic 
connections Jij axe build learning P — 2 phase-patterns at 
w'^ = 3Hz). 



VII. RELATIONSHIP OF THIS MODEL WITH 
THEORIES OF PATH-INTEGRATION 

Recently, path-integration system and the 
hippocampal-entorinal cortex circuit have been deeply 
investigated 0, [H, [H [11 HH, [H, HI, showing that 
place cells and grid cells form a map in which precise 
phase relationship among units play a central role to 
generate the spatial tuning. A number of models of 
the spatial firing properties of place and grid cells were 
offered. Generally two main categories are distinguished: 
models which focus on continuous attractor mechanisms 
and models which use interference between oscillators 
at dynamically modulated frequencies; however, deeper 
computational principles may exist that unify the 
different cases of neural integratio n 1261 . In oscillatory 
interference models [H [U, [2J, [H, |62| (see also for 
a review) the total synaptic input to a neuron (such 
as a grid cell or a place cell) is a weighted sum of 



the activities of n oscillatory inputs, whose oscillation 
frequency is modulated by the rat velocity and head 
direction. Grid and place cells, according to these mod- 
els, derive their temporal and spatial properties simply 
by detecting synchrony among such velocity-modulated 
oscillatory inputs. The oscillatory interference theory is 
one possible hypothesized mechanism of path integration 
and it has not been conclusively accepted or rejected. It 
is supported by recent studies suggesting that the pre- 
dicted velocity-modulated oscillators exist as theta cells 
(interneurons found throughout the septo-hippocampal 
circuit) whose inter-burst frequency shows a cosine 
modulation by running direction and a linear increase 
with running velocity 24 1. 

Such velocity-modulated oscillatory input was hypoth- 
esized to come from single oscillatory neurons [l9[, or 
networks such as subcortical "ring attractors" generat- 
ing velocity- modulated theta oscillations f23i |2^ 16^ . 

The properties of modulation and stability of fre- 
quency, and stability of multiple phase relationships, 
make our circuit a possible mechanism to build the 
velocity-modulated oscillators of the oscillatory interfer- 
ence theory. Indeed our circuit has a collective oscilla- 
tion frequency, which depends on the frequency stored 
in the connectivity matrix, and that can be modulated 
by changing the parameters such as 9th- Each neuron in 
the circuit has a phase determined by its position in that 
network, i.e. determined by the phase 0f of the stored 
phase-pattern. If the parameter 9th is modulated by an- 
imal speed, then the collective oscillation frequency of 
the circuit is modulated by the animal's speed, while the 
neurons preserve stable phase relationship among them. 
The other persistent-firing models [l^, [13, [62| of the os- 
cillators needed by the oscillatory interference theory suf- 
fer from problems related to robustness, as those en- 
countered by the single-cell oscillatory models [T^I, due 
to the variability in the frequency of persisting spik- 
ing [2^. Indeed the oscillatory interference models im- 
pose strict constraints upon the dynamical properties of 
the velocity-modulated oscillatory inputs, which have to 
preserve robust velocity-modulated frequency and stable 
phase relationships among them on relevant time scales 
in a manner robust to noise (many seconds, or dozens of 
theta cycle periods) [H, [H, [6l|, [zg . 
Notably, in our model, since connections Jij among units 
in the circuit are fixed by the learning rule ([7]), dynamics 
is a retrieval process and neurons preserve stable precise 
phase relationship among units and stable frequency even 
in noisy conditions, at least when the dynamics is in the 
basin of attraction of that phase relationship. 

Even in the more recent spiking models [H, [l^l of the 
oscillatory interference principle, in which many prob- 
lems related to noise are solved, the heterogeneity of pa- 
rameters of the cells which participate to the ring oscil- 
lator is not taken into account. 

Here we show that the circuit level interactions among 
units make the oscillation frequency and the phase- 
relationship of the system robust even with respect to 
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heterogeneity of the spiking thresholds of the units (see 
Sec. El) • 

This robustness, due to the proposed couphng which 
forces all the units of the circuit to have exactly the same 
period of oscillation and to have precisely the same phase 
relationships of stored pattern, may be useful in all cases 
of sequence coding. 

Moreover, our circuit can be easily programmed to 
cycle in different phase orders, by storing more than 
one phase-pattern as attractors. The circuit is a robust 
phase- shuffling ring oscillator, since the network has the 
capability of shuffling the order in which its neurons fire, 
by storing a variety of different phase-patterns within the 
connectivity. If the oscillators predicted by the interfer- 
ence theory can generate more than one phase sequence, 
as in the model presented here, then this could provide a 
potential mechanism to explain the phenomenon of hip- 
pocampus remapping [ttI . [78| . One of the more interesting 
discovery of place cells behavior is indeed the remapping 
of the place cell representation of space in response to 
a changes in sensory or cognitive inputs, i.e. place cells 
change their firing properties (place cells can appear dis- 
appear or move to other unpredictable locations). This 
change may be abrupt and similar to the switch from one 
attractor to another [73] ■ If the place cell will fire at a spe- 
cific place where its inputs become synchronized 0, [23l - 
[25} . by recalling a different phase-coded pattern among 
the ones stored in our circuit, it will change the phases 
of theta cells that are the inputs of the place cell, and it 
will change the specific 'place' where the inputs become 

Finally we note that even thou our model is not a con- 
tinuous attractor, it shares many similarity with such a 
class of models. Our model is a circuit with many dis- 
tinct attractors, one for each phase-relationship stored 
in the network, and the number of different attractor 
states is set by the maximal storage capacity studied 
here. Furthermore each attractor is a phase-coded pat- 
tern, replayed with a collective resonant frequency that 
can be modulated by changing for example the spiking 
threshold of the units. 

Even thou during exploration the activity of place 
cells may be explained by the superposition of velocity- 
controlled oscillators inputs, the recurrent connections 
inside the place cells network may have anyway a rele- 
vant role. 

During sleep, in absence of external input, the role of 
recurrent connections increases, probably due to an in- 
crease of excitability via neuromodulators or other mech- 
anisms, and the spontaneous activity of the network show 
temporarily short replay of stored patterns, probably ini- 
tiated simply by noise. 

So the pattern activated rcpeatly during experience, 
is stored in the connectivity, and then activated during 
sleep when the network is near a critic point and noise is 
able to initiate short replay sequences. 

Replay of phase-coded patterns of neural activity 
during sleep has been observed in hippocampus and 
neocortex H 



Notably in our model the time scale of reactivation 
is different from the time scale of storing, depends on 
the collective frequency which emerges from the connec- 
tivity, and may be accelerated or slowed down changing 
parameters such as spiking thresholds. Therefore, our 
model might be also relevant for replay in prefrontal cor- 
tex (PFC) or other cortical areas in which replay is ac- 
celerated with respect to awake activity. 

In the hippocampus, spikes representing adjacent place 
fields occur in rapid succession within a single theta cycle 
during behavior. Therefore, relative to this withintheta 
cycle rate, reactivation during sleep in hippocampus is 
not accelerated. However, reactivation in rat PFC is 
clearly compressed five to eight times relative to the wak- 
ing stated, II- 

Indeed, while in hippocampus one may 
think that the coding sequence is the within-theta cy- 
cle, in prefrontal cortex it is clearly seen that the cross- 
correlation among cells during sleep replay is time com- 
pressed compared with the cross-correlation during wak- 
ing state. The playback speed declines over time as does 
the strength of the replay, which is consistent for exam- 
ple with a simple increases of spiking threshold in our 
model. 



VIII. DISCUSSION 

We studied the temporal dynamics, including the stor- 
age and replay properties, in a network of spiking in- 
tegrate and fire neurons, whose learning mechanism is 
based on the Spike-Timing Dependent Plasticity. The 
temporal patterns we consider are periodic spike-timing 
sequences, whose features are encoded in the relative 
phase shifts between neurons. 

The importance of oscillations and precise temporal 
patterns has been pointed out in many brain structure, 
such as cortex [tI] , cerebellum [tiI, [t^ , or olfactory sys- 
tem [t^ ]. The proposed associative memory approach, 
with selective replay of stored sequence, can be a method 
for recognize an item, by activating the same memorized 
pattern in response to a similar input. Another possibil- 
ity is to have a way to transfer a memorized item to an- 
other area of the brain, such as for memory consolidation 
during sleep. During sleep, indeed, few spikes with the 
right phase relationship may initiate the retrieval of one 
of the patterns stored in the network and this reactivation 
may be useful for memory consolidation. The stored pat- 
tern is an attractor of the network dynamics, that is the 
dynamics spontaneously goes back to the retrieved phase- 
coded pattern for all the perturbations which leave the 
system within the basin of attraction. Therefore phase 
errors do not sum up, and the phase relationships may 
be transferred and kept stable over long time scales. 
The time scale of the pattern during retrieval, i.e. the pe- 
riod of oscillation l/i/, depends on (1) the time constants 
of the single neuron and Ts, (2) the spiking thresh- 
old 9th of the neurons, and (3) the connectivity, through 
the STDP learning shape A{t) and the time scale of the 
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pattern during learning mode Different areas of 

the brain may have different shape of STDP to subserve 
different oscillation frequencies and different functional 
role. Here we fix the shape of A{t) to the one observed 
in hippocampal cultures (see Fig. 1 and [4l|) and fo- 
cus on the dependence on the spiking threshold 6th of 
the neurons. The spiking threshold can modulate the 
frequency of the collective oscillation, leaving unaffected 
the phase relationships among the units. This opens a 
possible way to govern the frequency of collective oscil- 
lation via neuromodulators, and to encode information 
(such as velocity of the animal) in the frequency of the 
oscillations, in addition to the information encoded in 
the phase relationships. Notably, in a particular range 
of frequencies, the spiking threshold does not affect the 
frequency of oscillation but changes the number of spike 
per cycle during the retrieval dynamics. This means that 
information can be encoded via the number of spikes per 
cycle, independently from the information coded in the 
phase relationship among units, in agreement with the 
observations of independent rate and phase coding in 
hippocampus [l^]. Important to note, the phase rela- 
tionships and the frequency of the collective oscillation 
are both robust with respect to noise and to heterogene- 
ity of the spiking threshold of the units. 
A systematic study of the retrieval capacity of the net- 
work is proposed as a function of two parameters of the 
model: the frequency of the input pattern and the spiking 
threshold. The storage capacity, evaluated as Pmax/N, 
is always lower than the storage capacity of the Hop- 
field model. However, the information content of a single 
pattern in our dynamical model with N units is higher 
than the information content of a pattern in the Hopficld 
model with N-units. Indeed, an Hopficld pattern is a set 
of N binary values while our phase-coded pattern is a set 
of N real number (/)^ G [0, 2tt]. 

The role of STDP in the formation of sequences has 
been recently investigated in [65l . 1661] . These studies have 
shown how it's possible to form long and complex se- 
quences , but they did not concern themselves on how it's 
possible to learn and store not only the order of activa- 
tion in a sequence, but the precise relative times between 
spikes in a closed sequence, i.e. a phase-coded pattern. 
In our model not only the order of activation is preserved, 
but also the precise phase relationship among units. The 
tendency to synchronization of units is avoided in our 
model, without need to introduce delays or adaptation, 
due to the balance between excitation and inhibition that 
is in the connectivity of large networks when the phase 
coded pattern with random phases is learned using the 
rule in eqs. (5-8). In our rule all the connections, both 
positive and negative, scale with the time of presentation 
of patterns, keeping always a balance. Indeed, since (1) 
the stored phase are uniformly distributed in [0, 27r) and 
(2) the learning window has the property / A{T)dt — 0, 
then the connectivity matrix in eq. (7) has the prop- 
erty that the summed excitation (l/N) j. >o 



the summed inhibition (l/-^)X]ij <o"^«i equal in 
the thermodynamic limit (indeed they are of order unity, 
while their difference is of order 1/y/N). 

Under this conditions, we investigate how multiple 
phase-coded patterns can be learned and selectively re- 
trieved in the same network, as a function of time scale 
of patterns and network parameters. 

The task of storing and rccallingphase-coded memo- 
ries has been also investigated in [75[ in the framework 
of probabilistic inference. While we study the effects of 
couplings given by Eqn.(5) in a network of IF neurons, 
the paper [75| studies this problem from a normative the- 
ory of autoassociative memory, in which real variable Xi 
of neuron i represents the neuron i spike timing with re- 
spect to a reference point of an ongoing field potential, 
and the interaction H{xi, Xj) among units is mediated by 
the derivative of the synaptic plasticity rule used to store 
memories. 

The model proposed here is a mechanism which com- 
bines oscillatory and attractor dynamics, which may be 
useful in many models of path-integration, as pointed 
out in sec. (jVIip . Our learning model offers a IF cir- 
cuit able to keep robust phase-relationship among cells 
participating to a collective oscillation, with a modulated 
collective frequency, robust with respect to noise and het- 
erogeneities. Notably the frequency of the collective os- 
cillation in our circuit is not sensible to the single value 
of the threshold of each unit, but to the average value of 
the threshold of all units, since all units participate to a 
single collective oscillating pattern which is an attractor 
of the dynamics. 

Recently there is renewed interest in reverberatory 
activitv(81i] and in cortical spontaneous activitv[79l IsOj 
whose spatiotemporal structure seems to refiect the un- 
derlying connectivity, which in turn may be the result of 
the past experience stored in the connectivity. 
Similarity between spontaneous and evoked cortical ac- 
tivities has been shown to increase with age_|83|, and 
with repetitive presentation of the stimulus [82] ■ Inter- 
estingly, in our IF model, in order to induce spontaneous 
patterns of activity reminiscent of those stored during 
learning stage, few spikes with the right phase relation- 
ship are sufficient. It means that, even in absence of sen- 
sory stimulus, a noise with the right phase relationships 
may induce a pattern of activity reminiscent of a stored 
pattern. Therefore, by adapting the network connectivity 
to the phase-coded patterns observed during the learn- 
ing mode, the network dynamics builds a representation 
of the environment and is able to replay the patterns of 
activity when stimulated by sense or by chance. 

This mechanism of learning phase-coded patterns of 
activity is then a way to adapt the internal connectivity 
such that the network dynamics have attractors which 
represent the patterns of activity seen during experience 
of environment. 
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